Pitfalls of assessing extracted hierarchies for multi-class classification

Using hierarchies of classes is one the standard methods to solve multi-class classification problems. In literature, selecting right hierarchy considered play a key role in improving performance. Although different have been proposed, there still lack understanding what makes good and method extract perform better or worse. To this effect, we analyze compare some most popular approaches extracting hierarchies. We identify common pitfalls that may lead practitioners make misleading conclusions about their methods. address these problems, demonstrate using random an appropriate benchmark assess how hierarchy’s quality affects particular, show can become irrelevant depending on experimental setup: when powerful enough classifiers, final performance not affected by hierarchy. also comparing effect against non-hierarchical might incorrectly indicate superiority. Our results confirm datasets with high number generally present complex structures relate each other. datasets, dramatically improve